Information processing apparatus, information processing method, and recording medium

ABSTRACT

A plurality of projection images are obtained, on a basis of plural pieces of three-dimensional image data obtained by imaging an object at a plurality of different time points, by projection from a plurality of different projection directions corresponding to projection images in accordance with a three-dimensional partial region where a pixel value is increased and a three-dimensional partial region where the pixel value is decreased at a second time point with respect to a first time point among the plurality of time points, and the plurality of projection images are displayed on a display unit.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to an information processing apparatus, aninformation processing method, and a recording medium.

Description of the Related Art

When a doctor conducts a medical examination by using a medical image,to find a lesion site or carry out a follow-up, the doctor may observe aplurality of medical images obtained by imaging the same region at aplurality of time points. In this case, to emphasize a region where aclinical condition is changed, an image representing a differencebetween two three-dimensional images captured at different time points,that is, a temporal subtraction image may be created.

While projection processing is carried out to perform a projection froma certain direction with respect to the temporal subtraction imagecorresponding to a three-dimensional image, it becomes easier to graspthe region where the change has occurred. At the same time, in theprojection processing, the regions where the change has occurred may beoverlapped with each other on the projection direction in some cases.

As a technique for producing a differential image, US2002/0118868discusses a technique for producing a differential image by subtractingone image from another image.

SUMMARY OF THE INVENTION

According to some embodiments of the present invention, there isprovided an information processing apparatus including: an obtainingunit configured to obtain, on a basis of plural pieces ofthree-dimensional image data obtained by imaging an object at aplurality of different time points, a plurality of projection imagesobtained by projection from a plurality of different projectiondirections corresponding to projection images in accordance with athree-dimensional partial region where the pixel value is increased anda three-dimensional partial region where the pixel value is decreased ata second time point with respect to a first time point among theplurality of time points; and a display control unit configured todisplay the plurality of projection images on a display unit.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example of a system including an informationprocessing apparatus according to a first exemplary embodiment.

FIG. 2 illustrates an example of a hardware configuration of an imageprocessing apparatus and the information processing apparatus accordingto the first exemplary embodiment.

FIG. 3 is a flow chart illustrating an example of processing included inregistration processing according to the first exemplary embodiment.

FIG. 4 illustrates an example of differential processing according tothe first exemplary embodiment.

FIG. 5 illustrates an example of the differential processing accordingto the first exemplary embodiment.

FIG. 6 illustrates an example of the differential processing accordingto the first exemplary embodiment.

FIG. 7 illustrates an example of projection processing according to thefirst exemplary embodiment.

FIG. 8 illustrates an example of the projection processing according tothe first exemplary embodiment.

FIG. 9 illustrates an example of the projection processing according tothe first exemplary embodiment.

FIG. 10 is a flow chart illustrating an example of processing performedby the image processing apparatus according to the first exemplaryembodiment.

FIG. 11 is a flow chart illustrating an example of the processingperformed by the information processing apparatus according to the firstexemplary embodiment.

FIG. 12 illustrates an example of a screen displayed on a display unitof the information processing apparatus according to the first exemplaryembodiment.

FIG. 13 illustrates an example of the projection processing according tothe first exemplary embodiment.

FIG. 14 is a flow chart illustrating an example of the processingperformed by the image processing apparatus according to a secondexemplary embodiment.

FIG. 15 is a flow chart illustrating an example of the processingperformed by the information processing apparatus according to thesecond exemplary embodiment.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, exemplary embodiments of the present invention will bedescribed with reference to the drawings. Each of the embodiments of thepresent invention described below can be implemented solely or as acombination of a plurality of the embodiments or features thereof wherenecessary or where the combination of elements or features fromindividual embodiments in a single embodiment is beneficial.

First Exemplary Embodiment

An information processing apparatus according to a first exemplaryembodiment of the present invention obtains projection images projectedfrom a plurality of directions with regard to a temporal subtractionimage, based on plural pieces of three-dimensional image data obtainedby imaging an object at a plurality of different time points, anddisplays the projection images on a display unit.

FIG. 1 illustrates a system including the information processingapparatus according to the first exemplary embodiment. This systemincludes medical image capturing apparatuses 30, a PACS 40, an imageprocessing apparatus 50, and an information processing apparatus 60,which are connected to one another via a network 20.

The network 20 connects the respective apparatuses included in thissystem to one another. The network 20 is, for example, a local areanetwork (LAN). The network 20 is constituted, for example, by a devicesuch as a repeater, a hub, a bridge, or a router and a line such as theinternet.

The medical image capturing apparatus 30 is an apparatus configured tocapture a medical image used for a diagnosis. For example, the medicalimage capturing apparatus 30 is a magnetic resonance imaging (MRI)apparatus, an X-ray computed tomography (CT) imaging apparatus, or apositron-emission tomography (PET) imaging apparatus. A plurality ofmedical image capturing apparatuses 30 may be included in the system.

The PACS 40 refers to a picture archiving and communication system. ThePACS 40 receives and saves the image captured by the medical imagecapturing apparatus 30 and transmits the image to the respectiveapparatuses in accordance with requests of the connected apparatuses. Inaddition, the PACS 40 is provided with a data base that can save variousdata associated with the image together with the received image.

The image processing apparatus 50 is an apparatus configured to performimage processing according to the first exemplary embodiment. The imageprocessing apparatus 50 is, for example, an image processing workstation configured to process the medical image. The image processingapparatus 50 includes an obtaining unit 51, a generation unit 52, and anoutput unit 53. The obtaining unit 51 obtains an image on which theimage processing is performed in the image processing apparatus 50. Theobtaining unit 51 obtains the image from the medical image capturingapparatus 30 or the PACS 40 via the network 20. The generation unit 52performs the image processing according to the first exemplaryembodiment. The generation unit 52 includes a registration processingunit 521, a differential processing unit 522, a projection processingunit 523, and a composition processing unit 524. The registrationprocessing unit 521 performs processing of matching positions of theobjects included in the plural pieces of three-dimensional image dataobtained by imaging the object at the plurality of different time pointsto each other. The differential processing unit 522 performs processingof obtaining a difference between the three-dimensional image data inwhich the positions of the objects are matched to each other by theregistration processing unit 521. As a result, the temporal subtractionimage based on the plural pieces of three-dimensional image dataobtained by imaging the object at the plurality of different time pointsis obtained. The projection processing unit 523 obtains a projectionimage obtained by performing a projection from a certain direction withrespect to the temporal subtraction image obtained by the differentialprocessing unit 522. The projection processing unit 523 can obtain aplurality of projection images obtained by performing the projectionfrom a plurality of different projection directions with respect to onetemporal subtraction image. The projection processing unit 523 canperform, for example, maximum value projection processing and minimumvalue projection processing. The composition processing unit 524performs processing of combining the plurality of projection imagesobtained by the projection processing unit 523 to each other. The outputunit 53 outputs the plurality of projection images to the outside to bedisplayed on a display unit. For example, the output unit 53 outputs thecomposition image obtained by the composition processing unit 524. Theoutput unit 53 may also output the temporal subtraction image generatedby the differential processing unit 522 to the outside. Detaileddescriptions of the image processing according to the first exemplaryembodiment will be given below.

The information processing apparatus 60 is an apparatus configured toperform information processing according to the first exemplaryembodiment. The information processing apparatus 60 is, for example, anelectronic computer. A display unit 61 and an operation unit 62 may beconnected to the information processing apparatus 60. The display unit61 is, for example, a liquid crystal monitor. The operation unit 62 is,for example, a key board or a mouse. The display unit 61 and theoperation unit 62 may be integrated with a touch panel monitor.

The information processing apparatus 60 includes a control unit 63. Thecontrol unit 63 controls the respective apparatuses connected to theinformation processing apparatus 60 and the information processingapparatus 60 in an integral manner. The control unit 63 includes anobtaining unit 633, a display control unit 634, and a specification unit635. The obtaining unit 633 obtains the image displayed on the displayunit 61. The obtaining unit 633 obtains the image from the medical imagecapturing apparatus 30, the PACS 40, and the image processing apparatus50 via the network 20. The display control unit 634 controls a screendisplayed on the display unit 61. For example, the obtaining unit 633obtains the plurality of projection images obtained by the projectionprocessing unit 523. The display control unit 634 displays the pluralityof projection images obtained by the obtaining unit 633 on the displayunit 61. The specification unit 635 specifies a projection direction ofthe projection processing performed by the projection processing unit523. For example, the specification unit 635 specifies the projectiondirection on the basis of an operation input of a user via the operationunit 62. The processing performed by the information processingapparatus 60 will be described below.

FIG. 2 illustrates an example of a hardware configuration of the imageprocessing apparatus 50 and the information processing apparatus 60. Theimage processing apparatus 50 and the information processing apparatus60 include a central processing unit (CPU) 201, a random access memory(RAM) 202, a solid state drive (SSD) 203, a communication circuit 204,Universal Serial Bus (USB) 205, and High Definition Multimedia Interface(HDMI) (registered trademark) 206. The CPU 201 temporarily saves theprogram saved in the SSD 203 in the RAM 202. Furthermore, the CPU 201executes the program temporarily saved in the RAM 202 to control theapparatuses in an integral manner. The CPU 201 controls display of thedisplay unit 61. The RAM 202 is a primary storage device. The SSD 203saves a program used for operating the image processing apparatus 50 orthe information processing apparatus 60. The SSD 203 may be, forexample, another storage medium such as a flash memory. In addition, theSSD 203 saves data. The communication circuit 204 is a circuit connectedto the network 20 to perform communications with the respectiveapparatuses included in the system. The USB 205 and the HDMI (registeredtrademark) 206 are connection units. The USB 205 of the informationprocessing apparatus 60 is connected to the operation unit 62. The HDMI(registered trademark) 206 of the information processing apparatus 60 isconnected to the display unit 61.

Hereinafter, descriptions will be given of an example in which atomographic image obtained by the medical image capturing apparatus 30is used as the three-dimensional image data according to the firstexemplary embodiment. A specific example of the tomographic imageincludes an MR image obtained by the MRI apparatus, a CT image obtainedby the CT imaging apparatus, or a PET image obtained by the PET imagingapparatus. It should be noted that several different capturing methodsare proposed for the MR image. For example, it is possible to obtaintomographic images having different features such as a T1-weightedimage, a T2-weighted image, and a diffusion-weighted image. Thetomographic image is constituted by one or more tomographic plane imagescorresponding to two-dimensional images. While the tomographic planeimages at different capturing positions are laminated on each other, anobject such as a human body is three-dimensionally represented.

When a doctor conducts a diagnosis on whether or not an abnormality of apatient exists by observing the tomographic images, while an apparatusthat displays a tomographic plane image group constituting thetomographic images is used, an operation of searching for theabnormality is performed by switching the tomographic plane image to bedisplayed one by one. Since the tomographic images three-dimensionallyrepresent the object by laminating the tomographic plane imagescorresponding to the two-dimensional images on each other, it ispossible to identify coordinates of an arbitrary pixel by athree-dimensional Cartesian coordinate system. Specifically, the pixelis identified by coordinates (X, Y, Z) indicating the X-th pixel in thehorizontal direction and the Y-th pixel in the vertical direction of theZ-th tomographic plane image constituting the tomographic image. When alesion found by the doctor observing the tomographic image isrepresented, for example, “the lesion is present at the coordinates (X,Y, Z)” is recorded. In addition, an anatomical region such as a lesionregion or a “cerebrum region” can be identified by a pixel groupcorresponding to a coordinate group of the tomographic image.

Next, image processing performed by the registration processing unit 521will be described. The registration processing unit 521 performs imageregistration processing. The image registration processing refers toprocessing of deforming one or both of images such that the objects inthe two different images are matched to each other as much as possible.The doctor visually compares a tomographic image of a certain patientwith a tomographic image captured before the above-described tomographicimage and observes the tomographic images to determine whether or not aconcerning change occurs. At this time, even in the case of thetomographic images captured by the same medical image capturingapparatus 30, if an orientation of the patient at the time of thecapturing is varied, it may be difficult to compare the tomographicplane images with each other in some cases. The image registrationprocessing is performed on the newly captured tomographic image, and oneor both of the tomographic images are deformed to set shapes of thetomographic plane images to be similar to each other when the same siteis observed, so that the observation is facilitated. In a case whereboth the tomographic images are deformed, for example, the deformationprocessing is performed on the basis of a reference tomographic imageincluding the same imaging target corresponding to a reference apartfrom the two tomographic images. That is, the two tomographic images aredeformed such that each of the two tomographic images is matched to thereference tomographic image as much as possible.

In the deformation processing using the reference tomographic image, itis possible to reduce the number of times to perform the processing whenthe deformation is performed to match the plurality of tomographicimages to each other. For example, in a case where the deformation isperformed to match two arbitrary tomographic images out of fourtomographic images to be matched with each other as much as possible,maximum of six combinations exist. In a case where the registration ofall the tomographic images is performed on the basis of a method ofdeforming one of the tomographic images, the image registrationprocessing needs to be performed six times. In a case where theregistration of all the tomographic images is performed on the basis ofa method of using the reference tomographic image, it is sufficient thatthe image registration processing is performed four times. Furthermore,in the method of using the reference tomographic image, a user can startthe observation from the tomographic image on which the imageregistration processing has been performed, and it is possible to reducewaiting time.

In the image registration processing, deformation information forcontrolling the deformation of the tomographic image is generated. Then,the tomographic image is deformed in accordance with the deformationinformation. The deformation information refers to information formoving a pixel at certain coordinates. While the respective pixelsconstituting the tomographic image are moved in accordance with thedeformation information, the tomographic image after the deformation isobtained. In a case where a pixel that is not included in thedeformation information exists, a pixel in which a pixel value is notset is generated in the tomographic image after the deformation. In thiscase, with respect to the pixel in which the pixel value is not set, thepixel value can be set by a pixel value interpolation method such as alinear interpolation on the basis of a value and a position of the otherpixel in which the pixel value is set.

The registration processing unit 521 performs, for example, processingbased on a non-rigid registration. Herein, descriptions will be given ofimage registration processing using an algorithm called a largedeformation diffeomorphic metric mapping (LDDMM) (Miller, et al., 1993,Proceedings of the National Academy of Sciences of the United States ofAmerica, 90, 1 194-1 1948; Joshi et al., 1995, Geometric methods inApplied Imaging, San Diego, Calif.; Granander and Miller, 1996,Statistical computing and graphics newsletter 7, 3-8) as an example.

For example, the image registration processing of a first tomographicimage I(χ) and a second tomographic image J(χ) is performed. Herein, χdenotes a position vector represented by three-dimensional coordinates(x, y, z) and is equivalent to a voxel constituting respective latticepoints in the three-dimensional image. χ=(x, y, z)T is established, andT represents a transposition. In each of the first tomographic imageI(χ) and the second tomographic image J(χ), a family of sets Γ of pointscorresponding to an imaging target in the tomographic image={γ1, γ2, γ3, . . . , γn} is defined. The imaging target in the tomographic imageis, for example, a particular anatomical structure of the object.

It is assumed that, if the registration of the imaging targets of thefirst tomographic image I(χ) and the second tomographic image J(χ) isaccurately realized, a pixel group γ (γ belongs to Γ) corresponding tothe imaging target included in each of the tomographic image groupsappears at each of density values i and j with a joint probability P(i,j|γ) in I(χ) and J(χ). That is, a probability variable I or J, in whichthe density value of the pixel group γ corresponding to the imagingtarget included in each of the first tomographic image I(χ) and thesecond tomographic image J(χ) is set as each of the sample values, isset in each of the tomographic images. Then, it is assumed that amulti-dimensional probability vector constituted by a set of theprobability variables of the respective tomographic images, herein, atwo-dimensional probability vector (I, J) follows a predetermined jointdistribution (I(χ), J(χ)|γ).

Next, the first tomographic image I(χ) and the second tomographic imageJ(χ) on which the registration has not been realized, that is, the imageregistration processing has not been performed will be examined. Anon-rigid registration transformation for registering a first pixelgroup constituting the second tomographic image J(χ) with a second pixelgroup constituting the first tomographic image I(χ) will be representedas T(χ). If the first pixel group and the second pixel group areappropriately associated with the first tomographic image I(χ) and thesecond tomographic image J(χ) through this transformation T(χ), it isassumed that a two-dimensional vector (i, j) in which the respectivedensity values are set as a pair in the first pixel group and the secondpixel group follows the above-described joint distribution. A likelihoodfunction indicating a likelihood that the two-dimensional vector (i, j)is observed is defined by Expression 1 and Expression 2.

$\begin{matrix}\begin{matrix}{{P( {I,{J❘T}} )} = {\prod\limits_{\chi \in I}\;{P( {{I(\chi)},{J( {T(\chi)} )}} )}}} \\{= {\prod\limits_{\chi \in I}{\sum\limits_{\gamma \in \Gamma}\;{{P(\gamma)} \cdot {P( {{I(\chi)},{{J( {T(\chi)} )}❘\gamma}} )}}}}}\end{matrix} & (1) \\{{\sum\limits_{\gamma \in \Gamma}\;{P(\gamma)}} = 1} & (2)\end{matrix}$

It should be noted that the likelihood function of the above-describedformat is generally used in a likelihood analysis method, in particular,a maximum likelihood estimation method in the field of statisticalanalysis. In general, in the maximum likelihood estimation method, anaverage, a covariance, and the like in a population distribution are setto be undefined, and these are incorporated in the likelihood functionas population parameters to obtain a maximum likelihood estimator.According to the first exemplary embodiment, the population distributionis previously estimated, and a parameter φ of the non-rigid registrationtransformation T(χ) is incorporated in the likelihood function as apopulation parameter to obtain the maximum likelihood estimator. Theparameter φ will be described below. It should be noted that the maximumlikelihood estimation method can also be applied to the previouslyestimated population distribution.

The family of sets Γ of the anatomical structure corresponding to theimaging target in the tomographic image is constituted by two tissues,that is, target tissues corresponding to registration targets and theother surrounding tissues. Expression 1 and Expression 2 are transformedas follows.

$\begin{matrix}{{P( {I,J} )} = {\prod\limits_{\chi \in I}\;( {{{P(L)} \cdot {P( {{I(\chi)},{{J( {T(\chi)} )}❘L}} )}} + {{P(O)} \cdot {P( {{I(\chi)},{{J( {T(\chi)} )}❘O}} )}}} )}} & (3)\end{matrix}$

In addition, a maximum logarithm likelihood transformation T_(ML) isdefined by Expression 4.

$\begin{matrix}{T_{ML} = {\arg_{T}\mspace{11mu}\max{\sum\limits_{x}\;{\log( {P( {{I(\chi)},{J( {T(\chi)} )}} )} )}}}} & (4)\end{matrix}$

The likelihood function in Expression 4 is incorporated as a scale of asimilarity in an image registration evaluation function (objectivefunction) as defined by Expression 5.

$\begin{matrix}{{C(\Phi)} = {\sum\limits_{\chi}\{ {{- {C_{similarity}(\phi)}} + {\lambda\;{C_{smooth}(\phi)}}} \}}} & (5)\end{matrix}$

Where C_(similarity)(φ) denotes a similarity scale term defined as theabove-described likelihood. C_(smooth)(φ) denotes a smooth restrictionterm. λ denotes a weighting parameter to take a balance of the twoterms. C_(similarity)(φ) and C_(smooth)(φ) respectively denote thesimilarity scale term and the smooth restriction term in the respectivelocalized regions and are provided by Expression 6 and Expression 7.

$\begin{matrix}{\mspace{79mu}{{C_{similarity}(\phi)} = {\sum\limits_{\chi \in V}\;{\log( {P( {{I(\chi)},{J( {\chi + {\delta( {\chi;\phi} )}} )}} )} )}}}} & (6) \\{{C_{smooth}\;(\phi)} = {( \frac{\partial^{2}\phi}{\partial x^{2}} )^{2} + ( \frac{\partial^{2}\phi}{\partial y^{2}} )^{2} + ( \frac{\partial^{2}\phi}{\partial z^{2}} )^{2} + {2( \frac{\partial^{2}\phi}{\partial{xy}} )^{2}} + {2( \frac{\partial^{2}\phi}{\partial{xz}} )^{2}} + {2( \frac{\partial^{2}\phi}{\partial{yz}} )^{2}}}} & (7)\end{matrix}$

V denotes a localized region related to φ. Φ denotes a parameter fordescribing a registration transformation provided by Expression 8.T(χ)=χ+δ(χ;Φ)  (8)

δ(x; Φ) represents a deformation (FFD) of a free-form curved surfacedescribed by B-spline (curved surface). Φ represents a whole set ofB-spline control points. φ belongs to Φ and represents a subset ofcontrol points related to the respective localized regions.

To minimize the registration evaluation function defined in Expression5, for example, a steepest descent algorithm using a hierarchical gridwhich will be illustrated below is used. FIG. 3 is a flow chart forexemplifying processing included in the registration processingperformed by the registration processing unit 521. The processing basedon the steepest descent algorithm will be described with reference toFIG. 3.

In step S301, a control point Φ^(m) (m=0) is initialized. That is, aninitial value of Φ in the above-described processing is set.

In step S302, a gradient vector in Expression 5 is obtained on the basisof Expression 9.

$\begin{matrix}{{\nabla\; C} = \frac{\partial{C( \Phi^{T} )}}{\partial\Phi^{T}}} & (9)\end{matrix}$

In step S303, the control point Φ is updated in a range of ∥∇C∥>ϵ (ϵ isa small positive number) on the basis of Expression 10, and a gradientvector ∇C is obtained again. As a result, the control point Φ isupdated.

$\begin{matrix}{\Phi^{m + 1} = {\Phi^{m} + {\mu\frac{\nabla\; C}{{\nabla\; C}}}}} & (10)\end{matrix}$

In step S304, the hierarchy of the control point Φ is set to be finer.The flow proceeds to step S302 on the basis of Φ^(m+1) obtained fromExpression 10, and the above-described processing is repeated. It shouldbe noted however that Φ^(m) represents a B-spline control point grid atthe m-th hierarchical level.

The above-described processing from step S302 to step S304 is repeateduntil the highest hierarchical level is reached, and the processing iscompleted. A time point when the highest hierarchical level is reachedis when Φ converges. As an alternative to the above, the processing maybe completed after the processing is repeated a predetermined number oftimes.

It should be noted that the B-spline is used as the modeling method torepresent the free-form curved surface in the above-described example,but a Bezier method or other modeling methods may be used. In addition,the steepest descent method is used as the algorithm to minimize theregistration evaluation function in the above-described example, but anoptimization method such as a conjugate gradient method, aNewton-Raphson method, a quasi-Newton method, or a Levenberg-MarquardtMethod may be used.

Next, image subtraction processing in the image processing according tothe first exemplary embodiment will be described. As illustrated in FIG.4, the image subtraction processing refers to processing of subtractingpixel values from each other in the mutual pixels having thecorresponding position relationship in the two tomographic image groupsto obtain a subtraction image. For example, the image subtractionprocessing is performed by using the two tomographic image groups thathave the same imaging target and are captured at different times. As aresult, a subtraction tomographic image corresponding to an emphasizedimage in which a difference between the two tomographic images capturedat the different times, that is, a change is depicted, is obtained. Theabove-described subtraction tomographic image is referred to as atemporal subtraction image or a temporal subtraction tomographic image.It should be noted that, in the image subtraction processing accordingto the first exemplary embodiment, unless particularly mentioned, thepixel value group constituting the tomographic image having an oldercapturing time is subtracted from the pixel value group constituting thetomographic image having a newer capturing time.

In addition, in the tomographic image groups captured at the differenttimes, the position of the imaging target may be shifted in therespective tomographic images in some cases. Therefore, even in the caseof the tomographic image group captured by the same medical imagecapturing apparatus 30 and obtained on the basis of the same parametersetting, the temporal subtraction tomographic image in which only thepart where the change occurs is depicted may not be obtained in somecases by only subtracting the mutual pixels at the same coordinates fromeach other. For this reason, in a case where the image subtractionprocessing is performed with respect to the tomographic image groupscaptured at the different times, in order that the position relationshipof the imaging targets is matched in the tomographic image groups, theabove-described image registration processing is performed before theimage subtraction processing.

As a specific example, a case where two temporal subtraction tomographicimages as illustrated in FIG. 5 are obtained will be described. A firstterm on a left-hand side is a first CT image of a certain object, andthe imaging target is a lung. The first CT image illustrates a region ofan isolated lung cancer surrounded by an alveolar region. A second termon the left-hand side is a second CT image captured before the time whenthe first CT image is captured, and the imaging target is the lung. Thetemporal subtraction tomographic image is obtained by subtracting thesecond CT image from the first CT image. For simplicity, part where thechange occurs is only a lung cancer region in the first CT image and thesecond CT image.

First, the second CT image is deformed through the image registrationprocessing such that lung regions in the first CT image and the secondCT image are matched with each other. Next, the first CT image and thedeformed second CT image are subjected to the image subtractionprocessing to obtain the temporal subtraction tomographic image. As aresult, an image in which only the lung cancer region where the changeoccurs in the temporal subtraction tomographic image has a positivepixel value, and a pixel value of the other region is 0 is obtained. Ingeneral, the lung cancer region has a higher X-ray absorption rate thanthat of the surrounding alveolar region in the CT imaging. For thisreason, it is represented in the CT image that the pixel value of thelung cancer region is high, and the pixel value of the alveolar regionis low, for example. The region having the low pixel value where thelung cancer does not exist in the second CT image is subtracted from theregion having the high pixel value where the lung cancer exists in thefirst CT image. Therefore, only the region where the lung cancer existsis depicted to have the positive value in the temporal subtractiontomographic image illustrated in the right side of FIG. 5. Since it isassumed that parts other than the lung cancer region have no change, andthe pixel values thereof are matched to each other in the exampleillustrated in FIG. 5, the subtraction result of the mutual pixel valuesof the parts other than the lung cancer region is depicted as 0.

FIG. 6 illustrates another example of the temporal subtractiontomographic image. The first term on the left-hand side of FIG. 6 is thefirst CT image representing a state in which part of the bone isdissolved by an osteolytic cancer of the patient corresponding to theobject. The second term on the left-hand side of FIG. 6 is the second CTimage representing a state before the bone is dissolved which iscaptured before the first CT image. Similarly as in the above-describedexample, the second CT image is subtracted from the first CT image toobtain the temporal subtraction tomographic image as illustrated in theright side of FIG. 6. In the CT imaging, the bone region has a higherX-ray absorption rate than that of the region where the bone isdissolved. For this reason, it is represented in the CT image that thepixel value of the bone region is high, and the pixel value of theregion where the bone is dissolved is low, for example. Therefore, inthe temporal subtraction tomographic image, a region where the bone isdissolved and disappears is represented as a negative value where thepixel value is particularly low.

While the newly captured first tomographic image and the previouslycaptured second tomographic image are regarded as the targets, featuresof the temporal subtraction tomographic image obtained by the imagesubtraction processing are summarized from the above-described specificexample. When the region having the higher pixel value than that of thesecond tomographic image exists in the first tomographic image, thisregion is depicted as the region having the high pixel value (positivevalue) in the temporal subtraction tomographic image. On the other hand,when the region having the lower pixel value than that of the secondtomographic image exists in the first tomographic image, this region isdepicted as the region having the low pixel value (negative value) inthe temporal subtraction tomographic image.

Next, projection processing in the image processing according to thefirst exemplary embodiment will be described. The projection processingis processing of generating a projection image corresponding to atwo-dimensional image from the tomographic image corresponding to thethree-dimensional image. The projection image includes, for example, amaximum intensity projection (MIP) image, a minimum intensity projection(MinIP) image, a volume rendering image, a surface rendering image, orthe like. The MIP image is a maximum value projection image obtained bythe maximum value projection processing. The maximum value projectionprocessing is processing of displaying a maximum value in a projectionpath in an arbitrary view direction of the three-dimensional image on aproject plane. The MinIP image is a minimum value projection imageobtained by the minimum value projection processing. The minimum valueprojection processing is processing of displaying a minimum value in theprojection path in the arbitrary view direction of the three-dimensionalimage on the project plane.

In particular, since the MIP image generated from the temporalsubtraction tomographic image depicts the maximum pixel value in theview direction, it is effective to depict a region where the pixel valuebecomes higher than that of a previous tomographic image. For example,it is effective in a case where a lung cancer region or an osteoblasticcancer (cancer that causes the bone to be hardened) region newlyappears, or the like. In the tomographic image constituted by thetomographic plane image group illustrated in FIG. 7, when the MIP imageis generated in a case where a view direction vector is set as (0, 0,1), a two-dimensional image as illustrated in FIG. 8 is obtained. Inaddition, since the MinIP image generated from the temporal subtractiontomographic image depicts the minimum pixel value in the view direction,it is effective to depict a region where the pixel value becomes lowerthan that of the previous tomographic image. For example, it iseffective in a case where the osteolytic cancer or a crack or peelingregion such as a bone fracture newly appears or the like. In thetomographic image constituted by the tomographic plane image groupillustrated in FIG. 7, when the MinIP image is generated in a case wherethe view direction vector is set as (0, 0, 1), the two-dimensional imageas illustrated in FIG. 9 is obtained.

Next, the processing in the image processing apparatus 50 according tothe first exemplary embodiment will be described with reference to FIG.10. In the image processing apparatus 50, the obtaining unit 51 obtainsthe plurality of tomographic images, and the generation unit 52generates the temporal subtraction tomographic image and its projectionimage. Then, the output unit 53 outputs the projection image to theinformation processing apparatus 60. Furthermore, the generation unit 52generates the tomographic plane image corresponding to the coordinateson the basis of information for specifying coordinates of an arbitrarypixel which is input from the information processing apparatus 60, andthe output unit 53 outputs the tomographic plane image to theinformation processing apparatus 60. As a result, it is possible togenerate and output the appropriate image for the comparison and theobservation of the tomographic image. It should be noted that the outputunit 53 may output the temporal subtraction tomographic image to theinformation processing apparatus 60. Hereinafter, a case where the twotomographic images obtained while the same site of the same object isimaged by the same medical image capturing apparatus 30 are comparedwith each other for the observation will be described as an example.

In step S1001, the obtaining unit 51 obtains at least two or moretomographic image groups desired to be compared with each other for theobservation. Herein, the two tomographic images including the firsttomographic image and the second tomographic image previously obtainedby imaging the same object as the first tomographic image are obtained.The obtaining unit 51 obtains the tomographic image from the PACS 40 viathe network 20. As an alternative to the above, the obtaining unit 51obtains the tomographic image saved in the SSD 203 of the imageprocessing apparatus 50. The obtaining unit 51 obtains the tomographicimage in accordance with the operation input of the user. In anotherexample, the obtaining unit 51 performs control such that thetomographic image is automatically transmitted to the image processingapparatus 50 when the PACS 40 saves the tomographic image. Furthermore,control may be performed such that the obtaining unit 51 automaticallysearches the PACS 40 for the previous tomographic image of the sameobject to be transmitted to the image processing apparatus 50.Similarly, the obtaining unit 51 performs control such that thetomographic image may be automatically output to the image processingapparatus 50 when the medical image capturing apparatus 30 captures thetomographic image.

In step S1002, the registration processing unit 521 performs the imageregistration processing on the two tomographic images obtained in stepS101 to be deformed such that the two tomographic images are matched toeach other as much as possible. Herein, it is assumed that theregistration processing unit 521 deforms the second tomographic image toobtain a third tomographic image. That is, by the processing in stepS1002, the third tomographic image in which the position relationship ofthe imaging targets is substantially matched with the first tomographicimage is obtained. The output unit 53 outputs the third tomographicimage generated by the registration processing unit 521 to the SSD 203or the PACS 40 to be saved. For example, in a case where the processingillustrated in FIG. 10 is interrupted in step S1002, the obtaining unit51 obtains the third tomographic image from the SSD 203 or the PACS 40,and the subsequent processing can be resumed.

In step S1003, the differential processing unit 522 performs the imagesubtraction processing using the first tomographic image and the thirdtomographic image to generate the temporal subtraction tomographicimage.

In step S1004, the projection processing unit 523 identifies a pluralityof projection directions. The projection direction can be defined by aview-up direction vector and a view direction vector in athree-dimensional Cartesian coordinate system (X, Y, Z). For example, ina case where a direction in which Axial images are laminated is set as aZ axis, the view-up direction vector perpendicular to the laminationdirection of the tomographic plane images constituting the temporalsubtraction tomographic image is represented as (X, Y, Z)=(0, 0, −1).The projection image is obtained by projecting all the pixels with theview direction vector at (X, Y, Z)=(−sin θ, −cos θ, 0) in which θ is setto be higher than or equal to 0 and lower than 2π (π denotes a circleratio) while the view-up direction vector is maintained. In thisexample, when θ is 0, the projection image becomes a projection image ina Coronal image direction. When θ is π/2, the projection image becomes aprojection image in a Sagittal direction. According to the firstexemplary embodiment, the projection processing unit 523 generates aplurality of projection images by changing θ in the view directionvector at a constant interval. That is, when the projection images inthe plurality of projection directions generated by the projectionprocessing unit 523 are output to the display unit 61 of the informationprocessing apparatus 60, the projection images can be displayed as videoof rotation display.

Furthermore, in step S1004, the projection direction specified by theoperation input of the user is identified. The operation input of theuser is performed, for example, via the operation unit 62 of theinformation processing apparatus 60 and input to the image processingapparatus 50 by the specification unit 635 of the information processingapparatus 60. In addition, the projection direction can be previouslyset by the user. For example, in particular, the plurality of projectiondirections used when an input for specifying the projection directiondoes not exist are identified. When the input for specifying theprojection direction exists, the specified projection direction isidentified.

In step S1005, the projection processing unit 523 generates theprojection image on which the region where the pixel value is increasedis reflected. For example, the projection processing unit 523 generatesthe MIP image of the temporal subtraction tomographic image generated instep S1003. As a result, with regard to the plural pieces ofthree-dimensional image data obtained by imaging the object at theplurality of different time points, the projection image in accordancewith the three-dimensional partial region where the pixel value isincreased is obtained.

In step S1006, the projection processing unit 523 generates theprojection image on which the region where the pixel value is decreasedis reflected. For example, the projection processing unit 523 generatesthe MinIP image of the temporal subtraction tomographic image generatedin step S1003. As a result, with regard to the plural pieces ofthree-dimensional image data obtained by imaging the object at theplurality of different time points, the projection image in accordancewith the three-dimensional partial region where the pixel value isdecreased is obtained.

In step S1005 and step S1006, by generating the projection image onwhich the region where the pixel value is increased or decreased isreflected, it is possible to obtain the image where the influence ofartifact is reduced as compared with the temporal subtractiontomographic image. As a result, the user can more easily find out thepart where the change occurs when the user observes the image.

It should be noted that, when the position relationship of the imagingtargets of the first tomographic image and the third tomographic imagecompletely match, the pixel value of the pixel other than the regionwhere the change occurs in the temporal subtraction tomographic imagegenerated in step S1003 becomes 0. In actuality, a misalignment mayoccur in some cases even when the image registration processing isperformed. Therefore, in step S1005 and step S1006, when the projectionimage of the temporal subtraction tomographic image is generated, arough shape may also be depicted in some cases in the region other thanthe region where the change occurs.

In step S1007, the composition processing unit 524 combines theprojection images generated in step S1005 and step S1006 with eachother. As a result, it is possible to obtain the image representing thechange of the increase in the pixel value and the change of the decreasein the pixel value that can be observed at the same time. In step S1005and S1006, when the MIP image and the MinIP image are generated from thesame view direction, the images both having the same size, that is, thesame number of pixels are generated. In step S1007, the projection imagein which the pixel values of the mutual pixels at the same coordinate ofthe MIP image and the MinIP image obtained in step S1005 and step S1006are added to each other is generated.

In step S1008, the output unit 53 outputs the plurality of projectionimages including the composition projection image which are generated instep S1004 to step S1007 to the information processing apparatus 60.

The output unit 53 may output the temporal subtraction tomographic imagegenerated in step S1003 to the information processing apparatus 60 at apredetermined time point. The predetermined time point is, for example,step S1003. In another example, the predetermined time point is stepS1008, and the temporal subtraction tomographic image is output to theinformation processing apparatus 60 together with the plurality ofprojection images including the composition projection image.

That is, the image processing apparatus 50 according to the firstexemplary embodiment includes the obtaining unit 51 corresponding to anobtaining unit configured to obtain plural pieces of three-dimensionalimage data obtained by imaging an object at a plurality of differenttime points. In addition, the image processing apparatus 50 includes thegeneration unit 52 corresponding to a generation unit configured togenerate, on the basis of the plural pieces of three-dimensional imagedata, a plurality of projection images obtained by projection from aplurality of different projection directions corresponding to projectionimages representing a three-dimensional partial region where a pixelvalue is increased and a three-dimensional partial region where thepixel value is decreased at a second time point with respect to a firsttime point among the plurality of time points. Furthermore, the imageprocessing apparatus 50 includes the output unit 53 corresponding to anoutput unit configured to output the plurality of projection images tothe outside to be displayed on the display unit.

Next, the processing in the information processing apparatus 60according to the first exemplary embodiment will be described withreference to FIG. 11. In the information processing apparatus 60, theobtaining unit 633 obtains the projection image, and the display controlunit 634 displays the projection image on the display unit 61. Then, thespecification unit 635 specifies the projection direction. As a result,it is possible to generate and output the appropriate image for thecomparison and the observation of the image. In a case where thetemporal subtraction tomographic image is also output from the outputunit 53, the information processing apparatus 60 can display thetemporal subtraction tomographic image.

In step S1101, the information processing apparatus 60 accepts an inputfor instructing the image to be displayed on the display unit 61. Forexample, the information processing apparatus 60 accepts the operationinput of the user for instructing the image to be displayed on thedisplay unit 61 via the operation unit 62. The user can select andinstructs the image to be displayed on the display unit 61 from theimages saved in the SSD 203 of the information processing apparatus 60or the PACS 40. As an alternative to the above, while informationindicating that the image is captured by the medical image capturingapparatus 30 is input to the information processing apparatus 60, theinput is accepted as the instruction for displaying this captured image.

In step S1102, the obtaining unit 633 obtains the projection image fromthe image processing apparatus 50.

In step S1103, the specification unit 635 specifies the projectiondirection. According to the first exemplary embodiment, the plurality ofprojection directions are specified for rotation display as an initialsetting. In a case where the projection image corresponding to thespecified projection direction is not obtained in step S1102, thespecification unit 635 causes the obtaining unit 633 to obtain theprojection image from the image processing apparatus 50.

In step S1104, the display control unit 634 displays the projectionimages on the display unit 61. FIG. 12 illustrates an example of ascreen displayed on the display unit 61 in step S1104. In FIG. 12, theprojection images are displayed in a region represented as “ProjectionImage” on the right side. Herein, video of the rotation display isdisplayed. For example, with regard to the projection image of thetemporal subtraction tomographic image of the CT image in which a wholebody is imaged, the video of the display rotation is displayed where arough shape of the object continues rotating in a horizontal directionin which a vertical direction of the screen is set as a rotation axis.In the example illustrated in FIG. 12, since the human body looks likerotating, the shape of the human body is easily grasped. Furthermore, instep S1102, the composition image generated in step S1007 as illustratedin FIG. 10 is obtained and displayed on the display unit 61. That is,the plurality of projection images are displayed on the display unit 61.As a result, it is facilitated to grasp in which position of the humanbody the region displayed to be emphasized as the region having thechange exists.

In step S1105, the specification unit 635 determines whether or not thespecification of the projection direction exists. When the projectiondirection is specified, the flow proceeds to step S1103. When theprojection direction does not exist, the flow proceeds to step S1106.The user can specify the projection direction via the operation unit 62.According to the first exemplary embodiment, the video of the rotationdisplay in which θ in the view direction vector is changed at a constantinterval is displayed on the display unit 61 as the initial setting. Inthis case, for example, the specification unit 635 specifies θ at a timepoint when a mouse pointer indicating a specification position of amouse as an example of the operation unit 62 is overlapped with thedisplay part of the projection image as the projection direction. Instep S1103, θ at the time point when the mouse pointer is overlapped isspecified as the projection direction. In step S1104, the displaycontrol unit 634 displays the projection image corresponding to thespecified projection direction on the display unit 61. As a result, atthe time point when the mouse pointer is overlapped with the projectionimage, the rotation display is stopped. A region on the projection imagemay be clicked as an operation input for specifying the projectiondirection in a case where the rotation display is performed. Theprojection direction may also be specified by a wheel operation of themouse or a key operation of the key board on the projection image. Theuser can adjust the position of the rotation display by operating theview direction vector of the projection image.

In addition, in step S1105, a type of the projection image displayed onthe display unit 61 can be changed by the operation input of the user.For example, the user can display a desired projection image among theMIP image, the MinIP image, and the projection image obtained bycombining the MIP image and the MinIP image with each other on thedisplay unit 61. The information processing apparatus 60 may determinethe projection image to be displayed on the display unit 61 inaccordance with a purpose of capturing the tomographic imagecorresponding to the base of the projection image. For example, thedisplay control unit 634 displays an appropriate projection image fordetecting a lesion site related to a disease of the patientcorresponding to the object on the display unit 61. When the lung canceris suspected, the display control unit 634 displays the MIP image thatcan represent the region where the pixel value is increased on thedisplay unit 61.

In step S1106, the information processing apparatus accepts an operationinput for completing the display of the projection image on the displayunit 61. For example, the operation input for completing the display isperformed while an icon (not illustrated) displayed on the screen of thedisplay unit 61 by the display control unit 634 is operated. When theoperation input for completing the display exists, the display iscompleted. When the operation input for completing the display does notexist, the flow proceeds to step S1104, and the display of theprojection image continues.

A case is conceivable where, with regard to the projection imageobtained by combining the MIP image and the MinIP image with each other,the regions emphasized in the respective images are displayed to beoverlapped in a certain projection direction. For example, the caseincludes a case where the positive value recorded in the MIP image andthe negative value recorded in the MinIP image are overlapped with eachother when the change in which the pixel value is increased and thechange in which the pixel value is decreased occur on the same viewstraight line in a certain region as illustrated in FIG. 13. At thistime, the pixel values may be cancelled out during the addition processfor combining the MIP image with the MinIP image, and the pixel valuemay be close to 0 in the composition image in some cases. Even when achange occurs in the tomographic image group desired to be compared,there is a fear that a feature is not depicted in the projection image.In the processing in step S1102 to step S1104, for example, theprojection images in the plurality of projection directions aredisplayed as in the rotation display, so that it is possible to displaythe projection image from which the overlap of the emphasized regions iseliminated. The user can more accurately find out the region where thechange occurs. In addition, for example, as in the rotation display, byswitching the view direction of the projection image to be displayed, itis possible to alleviate labor for the user to manually switch the viewdirection.

The information processing apparatus 60 according to the first exemplaryembodiment includes the obtaining unit 633 corresponding to an obtainingunit configured to obtain, on the basis of plural pieces ofthree-dimensional image data obtained by imaging an object at aplurality of different time points, a plurality of projection imagesobtained by projection from a plurality of different projectiondirections corresponding to projection images in accordance with athree-dimensional partial region where a pixel value is increased and athree-dimensional partial region where the pixel value is decreased at asecond time point with respect to a first time point among the pluralityof time points. In addition, the information processing apparatusincludes the display control unit 634 corresponding to a display controlunit configured to display the plurality of projection images on adisplay unit.

That is, the information processing apparatus 60 according to the firstexemplary embodiment can three-dimensionally present the change betweenthe two or more tomographic image groups in a manner that the user caneasily grasp the change. The three-dimensional partial region where thechange occurs can be displayed such that the influence of the artifactis reduced. As a result, the user can more easily find out thethree-dimensional partial region where the change occurs. In particular,the region where the change occurs can be easily found out by theprojection image display of the temporal subtraction tomographic image,and the possibility that the lesion is overlooked can be reduced. Inaddition, the labor for visually searching for the lesion site can bealleviated.

Second Exemplary Embodiment

Next, a second exemplary embodiment of the present invention will bedescribed. According to the first exemplary embodiment, an example hasbeen described in which the maximum value projection processing or theminimum value projection processing is performed in the projectionprocessing. The information processing apparatus according to the secondexemplary embodiment obtains information of the region displaycorresponding to information regarding the region where the pixel valueis increased or decreased by an amount higher than or equal to athreshold or the region where a size is changed by an amount higher thanor equal to a threshold with regard to the plural pieces ofthree-dimensional image data captured at the different time points.

FIG. 14 is a flow chart for exemplifying the processing in the imageprocessing apparatus 50 according to the second exemplary embodiment.Since step S1001 to step S1004 illustrated in FIG. 14 are similar tostep S1001 to step S1004 illustrated in FIG. 10, the above-describedexplanation is equally applicable, and the detailed description will beomitted.

In step S1401, the projection processing unit 523 obtains the regionwhere the size is changed by the amount higher than or equal to thethreshold and the region where the pixel value is changed by the amounthigher than or equal to the threshold among the regions included in thetemporal subtraction tomographic image. The threshold can be previouslyset by the user. In addition, the threshold can be set in accordancewith a purpose of an examination for capturing a medical image. Herein,the size refers to the number of voxels or the volume in the case of thethree-dimensional image data. Information with regard to the regionobtained in step S1401 will be hereinafter referred to as regioninformation. The region information is considered to be useful todistinguish the region where the temporal change of the lesion regionoccurs from the other artifact. For example, when a certain cancerregion is enlarged over the temporal change, it is conceivable that theregion is changed so as to be expanded as a whole and distinguished fromthe artifact. In addition, when the cancer region is generated, thechange in the pixel value of the region is considered to be larger thanthat of the region where the artifact is generated.

In step S1402, the projection processing unit 523 generates a pluralityof projection images. The processing in step S1402 is similar to theprocessing in step S1004 to step S1007 illustrated in FIG. 10.

In step S1403, the output unit 53 outputs the region informationobtained in step S1401 and the plurality of projection images generatedin step S1402 to the information processing apparatus 60.

As a result, similarly as in the first exemplary embodiment, it ispossible to generate the projection image in which the number ofartifacts is low while the region where the change occurs is emphasized.

FIG. 15 is a flow chart for exemplifying the processing in theinformation processing apparatus 60 according to the second exemplaryembodiment. Since step S1101 to step S1106 illustrated in FIG. 15 aresimilar to step S1101 to step S1106 illustrated in FIG. 11, theabove-described explanation is equally applicable, and the detaileddescription will be omitted. In the processing of the informationprocessing apparatus 60 that will be described below, the projectionimage generated in the image processing apparatus 50 by the processingillustrated in FIG. 14 is used. The processing of the informationprocessing apparatus 60 illustrated in FIG. 15 may be applied to theprojection image generated in the image processing apparatus 50according to the first exemplary embodiment.

In step S1501, the display control unit 634 superimposes and displaysthe region display. First, the obtaining unit 633 obtains the regioninformation obtained in step S1401 illustrated in FIG. 14. The displaycontrol unit 634 superimposes and displays the display of the regionindicated by the region information on the projection image displayed onthe display unit 61 in step S1104. For example, the region where thetemporal change occurs is displayed to be emphasized by the MIP image orthe MinIP image.

In step S1502, the display control unit 634 determines whether or notthe region display superimposed and displayed in step S1501 isoverlapped in the projection image displayed in step S1104 from theprojection direction specified in step S1103. It is conceivable that theregion on which the region display is superposed is a region on whichthe user desires to pay attention such as a cancer region. For example,as illustrated in FIG. 13, the regions emphasized in both the MIP imageand the MinIP image may be overlapped with each other in a certainprojection direction in some cases. Furthermore, in a case where theregion emphasized in the MIP image or the MinIP image is overlapped withthe region that is superposed with the region display, it is conceivablethat the region on which the user desires to pay attention is overlappedin a certain projection direction. In a case where the region on whichthe user desires to pay attention is overlapped with the projectionimage displayed in step S1104, there is a fear that only one of them maybe depicted, or the pixel values are cancelled out and are not depictedas a result of the composition of the MIP image and the MinIP image.Thus, there is a fear that the region on which the user desires to payattention may be overlooked. In step S1502, in a case where the displaycontrol unit 634 determines that the overlap occurs, the flow proceedsto step S1503. In a case where the display control unit 634 determinesthat the overlap does not occur, the flow proceeds to step S1105.

In step S1503, the display control unit 634 displays an alert“overlapped” on the screen of the display unit 61. As a result, it ispossible to reduce the probability that the user overlooks the region onwhich the user desires to pay attention in the projection imagedisplayed on the display unit 61. When the display control unit 634displays the alert, the flow proceeds to step S1105.

Since step S1105 and step S1106 illustrated in FIG. 15 are similarprocessing as step S1105 and step S1106 illustrated in FIG. 11, theabove-described explanation is equally applicable, and the detaileddescription will be omitted.

In this manner, by the information processing apparatus according to thesecond exemplary embodiment, the region on which the user desires to payattention among the regions where the temporal change occurs in thethree-dimensional image data captured at the different time points canbe displayed in a more easily understandable manner. As a result, theprobability that the region on which the user desires to pay attentionis overlooked is reduced.

MODIFIED EXAMPLES

Next, a modified example of the first exemplary embodiment and thesecond exemplary embodiment will be described. FIG. 12 exemplifies thescreen on which the projection image is displayed by the display unit61, but the display control unit 634 may also display the tomographicimage or the tomographic plane image related to the projection image atthe same time. For example, FIG. 12 displays the tomographic planeimages at the position corresponding to the emphasized area in the“Projection Image”, that is, the region where the temporal changeoccurs, in a chronological order from the left. When the user specifiesthe position on the projection image via the operation unit 62, thedisplay control unit 634 can also display the tomographic plane imagescorresponding to the specified position. Specifically, the projectionimage is a two-dimensional image, and a Y coordinate corresponding tothe vertical direction of the pixel group constituting the projectionimage is equivalent to a Z coordinate of the tomographic image groupcorresponding to a generation source of the projection image. For thisreason, when coordinates corresponding to the display part of theprojection image are specified, the Y coordinate in the projection imageand the Z coordinate in the above-described tomographic image group areidentified. The display control unit 634 can display the tomographicplane image that passes through the identified Z coordinate of the firsttomographic image or the tomographic plane image that passes through theabove-described Z coordinate of the third tomographic image or displayboth of the images on the display unit 61. When the user discovers aregion where a lesion may exist, the user often observes the tomographicplane image at the same time. In the above-described case, it ispossible to alleviate the labor for switching the projection image andthe tomographic plane image or searching for the tomographic plane imageat a position corresponding to the lesion. That is, when the user usesthe mouse to click part displayed to be thickened up on the projectionimage based on the MIP image, for example, the screen can be instantlyswitched to the tomographic plane image where such a change occurs thatthe pixel value is increased to be observed. Thus, the labor for theswitching operation of the tomographic plane image in the related art isalleviated.

As another modified example of the first exemplary embodiment and thesecond exemplary embodiment, the display control unit 634 displays aplurality of tomographic plane images at the same time on the displayunit 61 or switches the plurality of tomographic plane images to bedisplayed on the display unit 61. In the tomographic image group such asthe first tomographic image, the second tomographic image, the thirdtomographic image, and the temporal subtraction tomographic image, thetomographic plane images (for example, Axial images or the like)constituting the above-described images are displayed in general. Thedisplay control unit 634 displays the tomographic plane imagesreconstituted to other tomographic plane of Coronal images, Sagittalimages, or the like at the same time or switches the tomographic planeimages to be displayed on the display unit 61 together with the Axialimages. Processing is performed such that the position relationship ofthe imaging targets of the first tomographic image and the thirdtomographic image is substantially matched by the image registrationprocessing. In a case where a display operation is performed on one ofthe tomographic images, a similar display operation is applied to theother tomographic image, so that the tomographic plane images at thesame position in both the tomographic images can be displayed whilebeing synchronized with each other. Furthermore, when a displayoperation with respect to a certain tomographic plane image is convertedto a display operation with respect to another tomographic plane imageto be applied, it is possible to display the other tomographic planeimages while being synchronized with each other or instantly switch thetomographic plane images to be displayed. That is, when the tomographicplane image including the lesion in the previous tomographic image isdisplayed, the tomographic plane image including the position where thelesion in the new tomographic image exists is automatically synchronizedand displayed. As a result, the user can easily check the change of thelesion, and the labor for the switching operation of the tomographicplane image is alleviated. In addition, with regard to the projectionimage at the same position, it is possible to change a display conditionsuch as a type of the projection image or the view direction by theoperation of the user. Each time the display condition is changed, theinformation processing apparatus 60 outputs the display condition to theimage processing apparatus 50. The image processing apparatus 50generates the projection image corresponding to the temporal subtractiontomographic image corresponding to the input the display condition to beoutput to the information processing apparatus 60.

The present invention can also be realized by processing in a mannerthat a program for realizing one or more functions of theabove-described exemplary embodiments is supplied to a system or anapparatus via a network or a storage medium, and one or more processorsin a computer of the system or the apparatus read out the program toexecute the processing. In addition, the present invention can berealized by a circuit that realizes one or more functions (for example,an application specific integrated circuit (ASIC)).

The information processing apparatus according to the above-describedrespective exemplary embodiments may be realized as a stand-aloneapparatus, and a mode in which a plurality of apparatuses are combinedwith each other so as to be mutually communicable to execute theabove-described processing may also be adopted, which are both includedin the exemplary embodiments of the present invention. A common serverapparatus or a server group may also execute the above-describedprocessing. It is sufficient when a plurality of apparatusesconstituting the information processing apparatus and an informationprocessing system are communicable to each other at a predeterminedcommunication rate, and the plurality of apparatuses do not necessarilyneed to exist in the same facility or the same country.

The exemplary embodiments of the present invention include a mode inwhich a program of software for realizing the above-described functionsof the exemplary embodiments is supplied to a system or an apparatus,and a computer of the system or the apparatus reads out and executes acode of the supplied program.

Therefore, to realize the processing according to the exemplaryembodiment by the computer, the program code itself installed into thecomputer is also one of the exemplary embodiments of the presentinvention. In addition, an operation system (OS) or the like running onthe computer performs part or all of the actual processing on the basisof an instruction included in the program read out by the computer, andthe above-described functions of the exemplary embodiments may also berealized by the processing.

A mode in which the above-described exemplary embodiments are combinedwith each other is also included in the exemplary embodiments of thepresent invention.

According to the exemplary embodiments of the present invention, whilethe projection image in accordance with the region where the changeoccurs between the three-dimensional image data captured at thedifferent time points is displayed, the region where the clinicalcondition is changed can be presented in an easily understandablemanner. In addition, while the plurality of projection images projectedfrom the plurality of different projection directions are displayed, theregion overlapped in a certain projection direction can be displayed inthe projection image from another projection direction. Thus, it ispossible to reduce the probability that the user overlooks the regionwhere the clinical condition is changed.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2015-171258, filed Aug. 31, 2015, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An information processing apparatus comprising:an obtaining unit configured to obtain, on a basis of plural pieces ofthree-dimensional image data obtained by imaging an object at aplurality of different time points, a plurality of projection imagesobtained by projection from a plurality of different projectiondirections corresponding to projection images in accordance with athree-dimensional partial region where a pixel value is increased and athree-dimensional partial region where the pixel value is decreased at asecond time point with respect to a first time point among the pluralityof time points; a display control unit configured to display theplurality of projection images on a display unit; and a specificationunit configured to specify a projection direction, wherein the displaycontrol unit displays the projection image obtained by projection in thespecified projection direction on the display unit.
 2. The informationprocessing apparatus according to claim 1, wherein the display controlunit switches and displays the plurality of projection images on thedisplay unit while the projection direction is changed.
 3. Theinformation processing apparatus according to claim 1, wherein thedisplay control unit switches and displays the plurality of projectionimages such that the projection direction rotates.
 4. The informationprocessing apparatus according to claim 1, wherein the projection imageobtained by the obtaining unit is a projection image obtained on a basisof a subtraction image based on the three-dimensional image dataobtained by imaging at the first time point and the three-dimensionalimage data obtained by imaging at the second time point.
 5. Theinformation processing apparatus according to claim 4, wherein theprojection image obtained by the obtaining unit is a projection imageobtained by combining a maximum value projection image obtained byperforming maximum value projection processing on the subtraction imagewith a minimum value projection image obtained by performing minimumvalue projection processing on the subtraction image.
 6. The informationprocessing apparatus according to claim 5, wherein the plurality ofprojection images include a projection image obtained by combining afirst maximum value projection image and a first minimum valueprojection image with each other which are obtained by performing themaximum value projection processing and the minimum value projectionprocessing on the subtraction image in a first direction, and aprojection image obtained by combining a second maximum value projectionimage and a second minimum value projection image with each other whichare obtained by performing the maximum value projection processing andthe minimum value projection processing on the subtraction image in asecond direction.
 7. The information processing apparatus according toclaim 4, wherein the display control unit displays at least one of theplural pieces of three-dimensional image data and the subtraction imagetogether with the projection image.
 8. The information processingapparatus according to claim 1, wherein the projection image obtained bythe obtaining unit is a projection image obtained on a basis of thethree-dimensional image data obtained by deforming at least one of thethree-dimensional image data obtained by imaging at the first time pointand the three-dimensional image data obtained by imaging at the secondtime point to perform registration of the three-dimensional image dataobtained by imaging at the first time point and the three-dimensionalimage data obtained by imaging at the second time point.
 9. Theinformation processing apparatus according to claim 1, wherein, in oneof the obtained projection images, in a case where at least either aregion where the pixel value is increased or decreased by an amounthigher than or equal to a threshold, or a region having a size largerthan or equal to a threshold among regions included in thethree-dimensional image data, is overlapped as viewed from theprojection direction of the one projection image, the display controlunit displays information indicating an overlap situation on the displayunit together with the one projection image.
 10. The informationprocessing apparatus according to claim 1, wherein the three-dimensionalimage data is a plurality of tomographic images of the object.
 11. Aninformation processing apparatus comprising: an obtaining unitconfigured to obtain plural pieces of three-dimensional image dataobtained by imaging an object at different time points; a generationunit configured to generate, on a basis of the plural pieces ofthree-dimensional image data, a plurality of projection images obtainedby projection in a plurality of different projection directionscorresponding to projection images in accordance with athree-dimensional partial region where a pixel value is increased and athree-dimensional partial region where the pixel value is decreased at asecond time point with respect to a first time point among the pluralityof time points; an output unit configured to output the plurality ofprojection images to an outside to be displayed on a display unit; and aspecification unit configured to specify a projection direction, whereinthe output unit outputs the projection image obtained by projection inthe specified projection direction to the outside.
 12. The informationprocessing apparatus according to claim 11, wherein the generation unitgenerates a subtraction image based on the three-dimensional image dataobtained by imaging at the first time point and the three-dimensionalimage data obtained by imaging at the second time point, and generatesthe projection image on a basis of the subtraction image, and whereinthe output unit outputs the subtraction image to the outside.
 13. Aninformation processing method comprising: obtaining, on the basis ofplural pieces of three-dimensional image data obtained by imaging anobject at a plurality of different time points, a plurality ofprojection images obtained by projection from a plurality of differentprojection directions corresponding to projection images in accordancewith a three-dimensional partial region where a pixel value is increasedand a three-dimensional partial region where the pixel value isdecreased at a second time point with respect to a first time pointamong the plurality of time points; displaying the plurality ofprojection images on a display unit; and specifying a projectiondirection, wherein the displaying displays the projection image obtainedby projection in the specified projection direction on the display unit.14. An information processing method comprising: obtaining plural piecesof three-dimensional image data obtained by imaging an object atdifferent time points; generating, on a basis of the plural pieces ofthree-dimensional image data, a plurality of projection images obtainedby projection from a plurality of different projection directionscorresponding to projection images in accordance with athree-dimensional partial region where a pixel value is increased and athree-dimensional partial region where the pixel value is decreased at asecond time point with respect to a first time point among the pluralityof time points; outputting the plurality of projection images to anoutside to be displayed on a display unit; and specifying a projectiondirection, wherein the outputting outputs the projection image obtainedby projection in the specified projection direction to the outside. 15.A non-transitory recording medium storing a program that when executedon a computer causes the computer to execute a process comprisingobtaining processing for obtaining, on a basis of plural pieces ofthree-dimensional image data obtained by imaging an object at aplurality of different time points, a plurality of projection imagesobtained by projection from a plurality of different projectiondirections corresponding to projection images in accordance with athree-dimensional partial region where a pixel value is increased and athree-dimensional partial region where the pixel value is decreased at asecond time point with respect to a first time point among the pluralityof time points, displaying processing for displaying the plurality ofprojection images on a display unit; and specifying processing forspecifying a projection direction, wherein the displaying processingdisplays the projection image obtained by projection in the specifiedprojection direction on the display unit.
 16. A non-transitory recordingmedium storing a program that when executed on a computer causes thecomputer to execute a process comprising obtaining processing forobtaining plural pieces of three-dimensional image data obtained byimaging an object at different time points, generation processing forgenerating, on the basis of the plural pieces of three-dimensional imagedata, a plurality of projection images obtained by projection from aplurality of different project directions corresponding to projectionimages in accordance with a three-dimensional partial region where apixel value is increased and a three-dimensional partial region wherethe pixel value is decreased at a second time point with respect to afirst time point among the plurality of time points, output processingfor outputting the plurality of projection images to an outside to bedisplayed on a display unit; and specifying processing for specifying aprojection direction, wherein the output processing outputs theprojection image obtained by projection in the specified projectiondirection to the outside.